Making users believe that they can interact with a virtual environment is important to user experience in a VR system. Users typically interact with a real environment using their hands, and to replicate this in the virtual environment, the VR system uses a controller, which can relay location and orientation information to the VR system, which uses different models, e.g., elbow model, arm model, etc., to render the user's arms/hands in the virtual environment. The VR system needs to know which hand is holding the controller to select the correct model, which is conventionally done by having the user manually provide information on the hand in which the user will be holding the controller.